I am a newbie on python pandas. I have a question on handling pandas dataframe. I use FRED (Federal Reserve Economic Data - St. Louis Fed) python api to get the big data. Below are the sources.
df_poverty = fred.search(search_str, order_by='title') # returns pandas dataframe
mask_poverty = df_poverty.title == search_str
df_poverty = df_poverty.loc[mask_poverty,['id']]
if not df_poverty.empty:
df_poverty_tmp = fred.get_series(df_poverty.iloc[0].id) # returns another pandas dataframe
print('******************')
print(df_poverty.index)
print('==================')
print(df_poverty.head())
print('==================')
print(df_poverty_tmp.index)
print('==================')
print(df_poverty_tmp.head())
The above codes print the following results.
******************
Index(['PPAAAR05000A156NCEN'], dtype='object', name='series id')
==================
id
series id
PPAAAR05000A156NCEN PPAAAR05000A156NCEN
==================
DatetimeIndex(['1989-01-01', '1990-01-01', '1991-01-01', '1992-01-01',
'1993-01-01', '1994-01-01', '1995-01-01', '1996-01-01',
'1997-01-01', '1998-01-01', '1999-01-01', '2000-01-01',
'2001-01-01', '2002-01-01', '2003-01-01', '2004-01-01',
'2005-01-01', '2006-01-01', '2007-01-01', '2008-01-01',
'2009-01-01', '2010-01-01', '2011-01-01', '2012-01-01',
'2013-01-01', '2014-01-01', '2015-01-01', '2016-01-01',
'2017-01-01', '2018-01-01'],
dtype='datetime64[ns]', freq=None)
==================
1989-01-01 17.9
1990-01-01 NaN
1991-01-01 NaN
1992-01-01 NaN
1993-01-01 18.9
dtype: float64
My target format of results are feature matrix like below with time-series indexing,
1989-01-01 PPAAAR05000A156NCEN 17.9
1990-01-01 PPAAAR05000A156NCEN NaN
1991-01-01 PPAAAR05000A156NCEN NaN
1992-01-01 PPAAAR05000A156NCEN NaN
1993-01-01 PPAAAR05000A156NCEN 18.9
I make python codes but the results are not satisfactory,
> df_poverty_tmp.append(df_poverty)
0 id
1989-01-01 00:00:00 17.7 NaN
1990-01-01 00:00:00 NaN NaN
1991-01-01 00:00:00 NaN NaN
1992-01-01 00:00:00 NaN NaN
1993-01-01 00:00:00 18.8 NaN
1994-01-01 00:00:00 NaN NaN
1995-01-01 00:00:00 17.6 NaN
1996-01-01 00:00:00 16.7 NaN
1997-01-01 00:00:00 16.2 NaN
1998-01-01 00:00:00 15.7 NaN
PPAAAL01000A156NCEN NaN PPAAAL01000A156NCEN
I want to know how to insert the pandas series value into the middle of pandas dataframe columns. Any reply will be thankful.